1. Field of the Invention
The present invention generally relates to data processing systems and more particularly relates to such systems that analyze survey data.
2. Description of the Prior Art
Many businesses have adopted the concept of Total Quality Management (TQM) to help improve profitability, ensure sustained customer loyalty, improve quality of products, etc. Originally developed by E. Deming and others, TQM is now widely accepted in many business circles throughout the world. A cornerstone of most total quality management (TQM) systems is the periodic measurement of identified parameters that relate to various aspects of the business. Through these measurements, managers can identify the areas within the organization that need improvement. Because the measurements are made on a xe2x80x9cperiodicxe2x80x9d basis, managers can gauge the effects of various changes made within the organization over time. The goal of most TQM systems is to make continuous improvement within the organization.
Businesses often generate and manage an array of measurements for purposes of analyzing the performance of individual operations in relationship to productivity, resource utilization, profitability, etc. One of the most important measurements in many TQM systems is that of customer satisfaction. To measure customer satisfaction, periodic customer feedback is required, usually obtained through customer surveys.
In many surveys, customers are asked a number of survey questions that are related to various products and/or services provided by a business. For many questions, the customers can respond both with a satisfaction rating and an importance rating. By analyzing the answers to these questions, managers can obtain insights into many areas, including customer loyalty, overall satisfaction, potential weakness, etc.
It is not uncommon for businesses to conduct their own customer surveys. However, it is increasingly popular to outsource this function to outside vendors who specialize in conducting customer surveys. Today, there are a number of vendors who conduct customer surveys, and provide a survey database to the subscribing business. One such company is Prognostics Inc., located in Menlo Park, Calif. An advantage of using such a vendor is that customer survey data for other companies, including competing companies, can often be obtained. This enables a business to perform comparisons between itself and its competitors.
Some of the vendors of customer surveys provide the survey data in electronic form, and often provide a software program for accessing the survey database. The software programs facilitate the generation of reports or the like, which can then be used by management. Prognostics is one such vendor. Prognostics provides survey data in a proprietary electronic format, and provides a software program called the Research Analysis Program (RAP) which can access the survey data. RAP can read the survey database, perform data requests, and provide a number of reports.
A difficulty with many of the software programs provided by customer survey vendors is that only certain data requests can be performed on the survey database. The performance of alternative data requests, which may be more desirable for a particular user, often require the user to execute several of the data requests offered by the vendor""s software program, and independently calculate the desired result using the data provided in the vendor""s reports. This may be particularly problematic because there are typically only a few xe2x80x9cexpertsxe2x80x9d within a business organization that have the knowledge necessary to select the appropriate data requests offered by the vendor""s software program, and perform the necessary calculations to produce the desired result. Thus, performing an analysis of a survey database provided by a survey vendor can require a significant amount of time of key personnel to arrive at the desired results.
This problem is magnified because the concept of Total Quality Management (TQM) requires that xe2x80x9ccontinuousxe2x80x9d measurement of selected parameters be performed. In practice, survey data is often taken once-per-week, once-per-month, once-per-year, etc., depending on the particular parameters to be measured. Customer satisfaction, for example, is often measured once-per-month or once-per-year, and requires a new customer satisfaction survey for each measurement. Every time a new customer satisfaction survey is performed, a significant amount of time of key personnel within an organization is often required to re-evaluate the survey database.
The present invention overcomes many of the disadvantages of the prior art by providing a method and apparatus for analyzing a survey database, wherein a user can perform a wider variety of requests to the survey database without having to independently calculate a desired result. Moreover, additional request types identified by xe2x80x9cexpertsxe2x80x9d within an organization are relatively easy to add to the system, and the parameters used by many of the requests can be set by the user at run time adding additional flexibility to the system.
In accordance with the present invention, these and other advantages are accomplished by using a rules-based expert system for forming and executing requests to a survey database. In a rules-based expert system, a number of rules are provided wherein the rules contain much of the xe2x80x9cknowledgexe2x80x9d of the experts, and the insights of xe2x80x9cdecision makersxe2x80x9d. This allows xe2x80x9cnon-expertxe2x80x9d users to perform xe2x80x9cexpertxe2x80x9d analysis of client satisfaction data in an accurate and repeatable fashion. This may make data analysis of a survey database faster, more reliable, and cheaper than in the prior art. Moreover, additional request types can be added to the system by simply adding additional rules, and linking the additional rules into the system.
Preferably, the rules-based expert system uses a number of predefined rules to process a request. Each rule is similar to a conditional statement in a conventional computer program (such as C++). However, the rules-based expert system is equipped with an inference engine, which is a special program for managing rules and applying them as appropriate. By contrast, a conventional program must indicate explicitly when a given conditional statement should be applied.
It is contemplated that the rules may have variables associated with them. By simply changing the values of the variables at run time, the analysis can be changed or optimized xe2x80x9con the flyxe2x80x9d. The underlying code may remain unchanged, and thus may not require recompilation, extensive debugging and/or quality assurance testing. In addition, the rules themselves may be added or changed, rather than just a variable associated with a rule, thereby changing the function of the rules-based expert system. Because the inference engine applies the rules as appropriate rather than in a predetermined order as in conventional code, the basic xe2x80x9calgorithmxe2x80x9d need not be modified when rules are added or modified.
In one embodiment of the present invention, a system is provided that has an interface module for accepting a request from a user, and a knowledge module that communicates with the interface module and is capable of accessing the survey database. The knowledge module processes the request at least in part by executing selected ones of a number of predefined rules. An inference engine selects which of the number of predefined rules are actually applied for a particular request.
The knowledge module may receive a request from the interface module, and determine which of the data elements in the survey database are required to derive the desired response to the user request. This may be accomplished, at least in part, by executing a number of predefined rules in the rules-based expert system, resulting in a number of selected data elements. The selected data elements may then be read from the survey database, and the knowledge module may determine the desired response to the user request at least in part by executing a number of predefined rules in the rules-based expert system. In most cases, the data elements within the survey database correspond to answers to questions in one or more customer surveys.
To determine the appropriate data elements in the survey database, the knowledge module may first determine the type of analysis requested by the user, and determine which of the data elements in the survey database are required to derive a response to the identified analysis type. Illustrative analysis types include a strength analysis type, a weakness analysis type, a threat analysis type, and an opportunity analysis type.
The present invention also contemplates providing caveats in response to certain requests. A first illustrative type of caveat is a request caveat. A request caveat may indicate if the request is a valid request. For example, the xe2x80x9cexpertsxe2x80x9d of a business organization may determine that a comparison of customer satisfaction between a mainframe computer system and a personal computer system does not provide any useful information. In this instance, this request may be deemed invalid. Preferably the system of the present invention still may process the request, but may issue a request caveat warning the user of any potential issues. It is also contemplated that a request caveat may indicate if any syntax errors have been found.
A second illustrative type of caveat is a results caveat. A results caveat indicates if the user should view the results with caution. In one example, the knowledge module may determine if the survey database has an insufficient number of selected data elements to derive a statistically significant response to the user request. That is, if there are insufficient data elements in the survey database to produce a statistically significant result, a results caveat may be provided indicating that the results should be viewed with caution. It is noted that the results may still be provided to the user. The results caveats may also identify a group of selected data elements that caused the result to be statistically insignificant. Because the rules in a rules-based expert system include knowledge provided by the xe2x80x9cexpertsxe2x80x9d, the request caveats may be provided by executing a number of user request rules provided in a rules-based expert system. The rules may not only determine if a request is valid, but may also indicate to the user the reasons why the user request was determined to be invalid.
Rather than submitting data requests directly to the survey database, it is contemplated that the knowledge module may assemble a number of data requests, and submit the data requests to a data module. Depending on the user request, different data requests may be required. For example, a request specifying an opportunity analysis type may require a strength analysis for a selected product and a weakness analysis for a competitor""s similar product. Each of these analysis types may require different data requests to the survey database. The knowledge module assembles the appropriate data requests, based in the request provided by the user, and provides the data requests to the data module.
The data module then provides the data request to a report generator. The report generator may be implemented in hardware or as a survey analysis program. The survey analysis program may be provided by the vendor of the survey data, or some other system or program that can access the survey database. This may be desirable when the survey data is in a proprietary format, thus requiring the use of the vendor""s survey analysis program.
In an illustrative embodiment, the report generator accesses the survey database and provides a number of intermediate reports to the rules-based expert system. The intermediate reports typically include a number of data objects therein. The rules-based expert system then uses the requested intermediate reports to calculate and/or assemble one or more reports that are provided to the user.
Preferably, the survey database includes a number of separate surveys. Each of the number of separate surveys may include data elements that correspond to a number of common questions, and a number of data elements that correspond to a number of questions that are not common to all of the separate surveys. Despite this lack of correlation between some of the separate survey files, it may be desirable to provide data requests and produce results that are based on two or more of the separate surveys. For example, one survey file may include survey data for one year while another survey file may include survey data for another year, and a comparison of the customer satisfaction between the two years may be desirable. Likewise, some of the questions in a given survey may be ordered differently or may be worded differently from another survey.
To overcome this and other difficulties, the present invention contemplates providing a correlation table which correlates the various questions in the separate survey files. This is preferably accomplished by correlating each question in each survey file to a master listing of generic questions or topics. Alternatively, each question may be correlated to all other similar questions in all other survey files.
Preferably, the interface module displays a listing of the generic questions, which may be selected by the user when forming a user request. To determine the appropriate generic questions for a selected survey, the interface module may access the correlation table, preferably via the data module, and retrieve the generic questions that correspond to the actual questions in the selected survey. Once a user request is formed, the knowledge module may identify the actual question numbers and the appropriate surveys that correspond to the generic questions provided in the user request. The data requests assembled by the knowledge module preferably specify the actual questions numbers, and the corresponding survey names. Thus, the data requests provided by the knowledge modules preferably identify selected data elements within the survey database for analysis.
In some cases, it is desirable to provide ordered processing within a rules-based expert system. This can be difficult when using some conventional expert type systems. To overcome this limitation, the present invention contemplates identifying a first set of rules and a second set of rules that are required for performing a task, wherein the first set of rules are to be processed before the second set of rules. The first set of rules are submitted for processing, preferably by an inference engine of the rules-based expert system, and a first result is provided. Thereafter, the second set of rules are submitted to the inference engine. Thus, a method is provided for producing ordered processing within a rules-based expert system.
Preferably, when the first set of rules are processed by the inference engine, a number of first agenda objects are generated within an object oriented database. Each of the agenda objects may specify a data request and a number of rules. The system then executes each of these agenda objects in a predetermined order. For example, the system first processes a first agenda object. This includes executing the data request associated with the first agenda object, and obtaining the appropriate data elements from the survey database. Thereafter, the number of rules associated with the first agenda object are executed using the inference engine. This provides a number of first results. The subsequent agenda objects are processed in a similar manner. Using this approach, ordered processing can be achieved in a rules-based expert system.
Finally, it is contemplated that a scripting feature may be provided. As indicated above, Total Quality Management (TQM) often requires periodic evaluation of survey data. Because it may be desirable to make selected requests to a number of survey databases, including subsequent survey databases obtained at a later time, it may be desirable to save a number of requests to a script file. The requests in the script file can then be executed automatically by simply executing the script when subsequent survey databases are received.
In many cases, the same basic requests are executed on each survey database received. However, selected parameters or system settings may change over time. For example, the definition of a strength may be defined differently over time, possibly requiring a modification to a parameter or system setting. As indicated above, modifying selected parameters may be automatically reflected in the rules that reference the parameter.
To facilitate the execution of requests stored in a script file using alternative parameter or system settings, the present invention contemplates saving selected user requests independent of the system settings and/or parameters in a user request script. The user request script can then be read, and processed using new parameter or system settings that are currently active at the time of execution. The term xe2x80x9csystem settingsxe2x80x9d may include adding, modifying or deleting rules from the rules-based expert system.